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ABSTRACT 


-Broadcast  in  a  coimwnl cation  network  Is  the  delivery  of  copies  of 
messages  to  all  nodes.  A  broadcast  algorithm  is  reliable  if  all  messages 
reach  all  nodes  in  finite  time,  in  the  correct  order  and  with  no  duplicates. 
The  present  paper  presents  an  efficient  reliable  broadcast  algorithm. 
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1.  INTRODUCTION 

Broadcast  multipoint  communication  Is  the  delivery  of  copies  of  a 
message  to  all  nodes  In  a  comnunl cation  network.  In  a  network  with  mobile 
subscribers,  for  exanple,  the  location  and  connectivity  to  the  network  of 
such  subscribers  may  change  frequently  and  this  Information  must  be  broad¬ 
cast  to  all  nodes  In  the  network,  so  that  the  corresponding  directory 
list  entry  can  be  updated.  Broadcast  messages  are  used  In  many  other 
situations,  like  locating  subscribers  or  services  whose  current  location 
Is  unknown  (possibly  because  of  security  reasons),  updating  distributed 
data  bases  or  transmitting  Information  and  commands  to  all  users  connected 
to  the  communication  network. 

There  are  certain  basic  properties  that  a  good  broadcast  algorithm 
must  have  and  the  most  Important  are:  a)  reliability,  b)  low  communication 
cost,  c)  low  delay,  d)  low  memory  requirements.  Reliability  means  that 
every  message  must  indeed  reach  each  node,  duplicates.  If  they  arrive  at  a 
node,  should  be  recognizable  and  only  one  copy  accepted,  and  messages  should 
arrive  In  the  same  order  as  transmitted.  Communication  cost  Is  the  amount 
of  comnunl cation  necessary  to  achieve  the  broadcast  and  consists  of,  first, 
the  number  of  messages  carried  by  the  network  per  broadcast  message,  (broad¬ 
cast  communication  cost),  second,  the  number  of  control  messages  necessary 
to  establish  the  broadcast  paths,  (control  comunlcatlon  cost),  and,  third, 
the  overhead  carried  by  each  message  (overhead  cost),  low  delay  and  memory 
are  basic  requirements  for  any  communication  algorithm,  and  broadcasts  are 
no  exception. 


The  broadcast  communication  cost  Is  minimized  If  the  algorithm  uses 
spanning  trees,  but  normally  tnere  Is  need  for  a  large  control  communication 
cost  In  order  to  establish  and  maintain  these  trees.  However,  the  control 
cost  can  be  reduced  considerably  provided  that  the  routing  mechanism  In 
the  network  constructs  routing  paths  that  form  directed  trees  towards 
each  destination.  In  which  case  these  trees  can  be  used  In  the  reverse 
direction  for  broadcast  purposes.  This  general  Idea  Is  presented  In  [1], 
but  the  authors  show  that  the  proposed  algorithms  named  reverse  path 
forwarding  and  extended  reverse  path  forwarding  are  not  reliable  when  the 
routing  algorithm  Is  dynamic,  since  In  this  case  nodes  may  never  receive 
certain  messages,  duplicates  may  be  received  and  accepted  at  nodes,  and  the 
order  of  arriving  messages  may  not  be  preserved.  As  said  before.  In  order 
to  be  efficient,  the  above  mentioned  algorithms  require  that  the  routing 
paths  to  each  destination  are  directed  trees.  An  adaptive  routing  algorithm 
that  maintains  at  all  times  spanning  directed  trees  rooted  at  the  destina¬ 
tion  has  been  proposed  In  [2]  and  throughout  the  present  paper  we  assume 
that  the  protocol  of  [2]  Is  the  underlying  routing  algorithm  In  the  network. 
However,  for  the  reasons  stated  before,  namely  the  fact  that  the  routing 
paths  are  dynamic,  the  broadcast  algorithm  of  [1]  Is  unreliable  even  if 
applied  to  the  routing  procedure  of  [2], 

The  purpose  of  the  present  paper  Is  to  propose  and  validate  an 
algorithm  whose  main  property  Is  that  the  broadcast  propagating  on  the 
tree  provided  by  the  routing  protocol  of  [2]  Is  reliable.  It  Is  convenient 
for  the  purpose  of  our  discussion  to  separate  the  property  of  reliability 
Into  two  parts:  completeness  means  that  each  node  accepts  broadcast 
messages  In  the  order  released  by  their  origin  node,  without  duplicates 
or  messages  missing,  while  finiteness  Is  the  property  that  each  broadcast 
message  Is  Indeed  accepted  at  each  node  In  finite  time  after  Its  release. 


t 


As  mentioned  and  exemplified  by  the  authors,  the  algorithms  of  [1]  are 
neither  complete  nor  finite.  In  the  algorithm  of  the  present  paper, 
completeness  Is  achieved  by  requiring  nodes  to  store  broadcast  messages 
In  the  memory  for  a  given  period  of  time  and  by  introducing  counter  nunbers 
at  the  nodes.  Finiteness  Is  obtained  by  attaching  a  certain  Impeding 
mechanism  to  the  routing  protocol.  We  may  mention  here  that  It  Is  relativ¬ 
ely  easy  to  make  a  broadcast  algorithm  reliable  If  one  allows  Infinite 
memory,  unbounded  counter  numbers  and  Infinite  overhead  in  the  broadcast 
messages.  However,  the  properties  that  make  our  algorithm  tractable  are: 
bounded  memory,  bounded  counter  numbers,  no  overhead  carried  by  broadcast 
messages  (In  form  of  counter  numbers  or  any  other  kind)  and  the  fact  that 
the  Impeding  mechanism  Is  not  activated  most  of  the  time. 

In  the  rest  of  the  paper  we  proceed  as  follows:  Sec.  2.1  contains 
a  brief  description  of  the  routing  algorithm  of  [2].  Sec.  2.2  and  2.3 
build  the  reliable  broadcast  protocol  step  by  step,  while  Its  final  form 
and  main  properties  are  given  In  Sec.  3.  The  proofs  of  the  main  theorems 
are  Included  In  the  Appendix. 


T 


-  4  - 

2.  THE  BROADCAST  PROTOCOL 
2.1  The  Routing  Protocol 

The  underlying  routing  protocol  considered  In  this  paper  Is 
The  Basic  Protocol  of  [2].  In  suamary,  this  protocol  proceeds  In  updating 
cycles  triggered  and  terminating  at  the  destination  node  named  SINK. 

An  updating  cycle  consists  of  two  phases:  a)  control  messages  propagate 
uptree  front  SINK  to  the  leaves  of  the  current  tree  and  each  node  1  per¬ 
forms  this  phase  whenever  It  receives  a  control  message  MSG  from  Its 
current  preferred  neighbor  p^;  b)  control  messages  propagate  downtree, 
while  new  preferred  neighbors  are  selected  and  this  phase  Is  performed  at 
node  1  upon  detecting  receipt  of  MSG  from  all  neighbors. 

In  the  following  sections  we  shall  need  to  Identify  the  updating 
cycles  and  It  Is  convenient  to  attach  to  each  cycle  a  counter  number  a. 

For  the  time  being  a  will  be  unbounded,  but  later  we  shall  show  that 
a  binary  variable  Is  sufficient.  The  routing  protocol  is  specified  by 
the  following  algorithm. 

Routing  Algorithm  for  node  1  (RA) 

1.  For  MSG  [l, a) 

Z.  H^l)  -  1 

3.  If  l  ■  Pj,  then:  ♦  a;  send  MSGUj)  to  all  l  €  G^ ,  except  p^ 

4.  If  v£‘  e  Gj,  holds  N^(£')*l,  then:  send  MSG(a^)  to  p^ ;  select 

new  p^;  vl'  6  G^,  set  NjU')»0. 

Here  G^  denotes  the  set  of  neighbors  of  1  and  "For  MSG  (£,<*)  "mean^1  when 

receiving  MSG(o)  from  neighbor  f,  perform . Subscript  1  Indicates 

variables  at  node  1  and  corresponding  variables  without  subscript  Indicate 
variables  In  the  received  message.  He  have  deliberately  suppressed  from 
the  algorithm  of  [2]  all  variables  that  are  not  directly  relevant  to  the 
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broadcast  (like  d^,  d,  D^U))  and  have  not  explicitly  Indicated  the  proce¬ 
dure  for  selecting  the  new  p,  because  it  Is  not  important  for  our  purpose, 
except  for  the  property  that  It  maintains  at  all  times  a  directed  spanning 
tree  rooted  at  SINK.  For  simplicity  Pj  will  be  called  the  father  of  1. 

The  algorithm  Is  indicated  for  a  given  SINK  that  Is  not  specified  explicitly 
(again  for  simplicity  of  notation)  and  that  becomes  the  source  of  the  broad¬ 
cast  messages.  The  SINK  performs  the  following  algorithm  (lines  are  num¬ 
bered  to  match  equivalent  Instructions  in  the  Routing  Algorithm): 

3.  Start  new  cycle  by  aSINK  “sink*!*  send  M^aSINK^  t  €  GSINK. 

(Note:  <  3  >  can  be  performed  only  after  <  4  >  of  the  previous 
cycle  has  been  performed). 

1.  For  MSG(o) 

2*  NSINK^'"1, 

4.  IX  €  GSINK*  ho^ds  NSINK^ then  cycle  a  completed, 

wt'  €  gSINK’  set  NSINK^')“°' 

In  principle,  the  routing  tree  can  be  used  for  broadcast  purposes 
as  follows:  a  node  i  accepts  only  broadcast  messages  received  from  Its 
father  p^  and  forwards  them  to  all  nodes  k  whose  father  Is  i.  Observe 
that  we  distinguish  between  receiving  a  broadcast  message  and  accepting  it. 

In  general,  a  broadcast  message  received  at  a  node  may  be  either  accepted 
or  rejected,  depending  on  the  specific  algorithm. 

The  first  problem  that  one  encounters  with  the  above  procedure  Is 
that  In  the  routing  algorithm  a  node  1  knows  only  Its  father  p^ ,  but 
does  not  know  the  nodes  k  for  which  Pj«1.  Consequently,  we  need  an 
addition  to  the  routing  algorithm,  so  that  whenever  a  node  1  changes  Its 
father  Pj,  (line  <4  >  In  the  Routing  Algorithm)  It  sends  two  special 
messages:  DCL  (declare)  to  the  new  father  and  CNCL  (cancel)  to  the  old  father. 

*A  specific  line  In  an  algorithm  will  be  indicated  in  angular  brackets  <>. 
The  algorithm  we  refer  to  will  either  be  clear  from  the  context  or  indicated 
explicitly. 


Each  node  1  will  have  a  binary  variable  z^(k)  for  each  neighbor  k 
that  will  take  on  th  value  1  If  1  thinks  that  pk*i  and  0  otherwise. 
Receipt  of  DCL  at  node  k  from  1  shows  that  at  the  time  OCL  was  sent, 
node  1  selected  k  as  p^,  so  that  2^(1)  Is  set  to  1.  The  nodes  1 
for  which  zk(i)*l  are  called  sons  of  k.  Observe  that  because  of  link 
delays,  if  1  Is  a  son  of  k  It  does  not  mean  that  at  the  same  time  k 
Is  the  father  of  1.  We  can  now  write  in  our  notation  the  combination 
of  the  above  routing  algorithm  and  the  Extended  Reverse  Path  Forwarding 
(ERPF)  Broadcast  Algorithm  of  [1],  where  B  denotes  a  broadcast  message: 

ERPF  Broadcast 
1.  For  MSG  U, a) 


2. 

MiU)  -  1 

3. 

it  ^  *  P-j ,  then :  send  MSG  to  all  1  z  G^, 

except  pi 

4. 

if  V4*  €  Gj,  holds  (£')-!,  then: 

4. a. 

select  new  p^; 

4.b. 

If  new  Pj  f  old  p^ ,  then  send  DCl(a)  to 

new  p.|  and  CNCL 

to  old  p.| ; 

4.c. 

send  MSG(a}  to  old  ;  vl*  €  Gj  •  set  «•  0. 

5. 

For  CNCLU)  set  ZjU)  -  0. 

6. 

For  0CL(£,a),  set  z^U)  ♦  1 

7. 

For  B (l) 

8. 

ff  l  •  p^ ,  then  accept  B ,  send  copy  of  B  to  all 

I's.t.  z1(t')=l 

Note:  It  Is  worth  pointing  out  that  line  <  8  >  means  that  1  ft*  p^, 
then  B  Is  accepted,  while  If  l  f  p^,  then  It  Is  rejected. 
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2.2  Completeness 

As  mentioned  In  the  introduction,  the  above  broadcast  protocol 
is  noncomplete  and  nonfinite.  The  purpose  of  this  section  is  to  show  that 
completeness  can  be  achieved  by  using  memory  and  counter  numbers  at  the 
nodes.  Observe  that  we  achieve  our  goal  without  requiring  that  the  counter 
numbers  will  be  carried  by  the  broadcast  messages,  so  that  the  algorithm 
has  no  overhead  cost.  For  purposes  of  illustration,  it  is  best  to  impose 
for  the  time  being  no  bounds  on  the  memory  or  on  the  counters  and  also  to 
describe  the  protocol  as  if  completeness  was  already  proved.  After  indica¬ 
ting  the  formal  algorithm  we  shall  show  that  it  is  indeed  complete  and  in 
the  following  sections  we  shall  introduce  features  that  will  make  the 
memory  and  the  counters  finite. 

Suppose  we  require  each  node  1  to  have  a  LIST^  where  every  accepted 
broadcast  message  is  stored  in  the  received  order  and  also  to  keep  a  counter 


ICj,  counting  the  accepted  messages.  Completeness  of  the  broadcast  protocol 
means  that  for  any  value  of  IC^,  the  list  LISTi  contains  all  messages  sent 


by  the  source  SINK  up  to  counter  number  IC. ,  with  no  duplicates  and  in  the 

D 

correct  order.  In  other  words  if  IC^  denotes  the  value  of  IC^  after  broad¬ 


cast  message  B  was  accepted  at  node  i,  we  have  IC^  =  for  all  B  and  all 


In  the  algorithm  we  also  require  that  every  DCL  message  sent  by  node  k  will 


have  the  format  DCL(a.IC)  where  IC  *  IC^  at  the  time  DCL  is  sent.  In  this 


way  when  a  node  i  receives  DCL  from  k,  it  will  have  updated  information 


about  the  "state  of  knowledge",  denoted  by  IC^(k),  of  its  new  son  k.  Only 

D 

broadcast  messages  B  with  IC^  >  IC^(k)  need  to  be  sent  by  i  to  k. 

The  formal  algorithm  is  now 
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The  Complete  Routing  -  Broadcast  Algorithm  (CRB)  for  node  i 

1.  For  MSG(z,a) 

2.  N^O-1 

3.  i_f  i=p. ,  then  :  a;  send  MSG(a.. )  to  all  i'eG^  except  p^ 

4.  if  Vt'eGi,  holds  N^f)  -  1,  then; 

4a.  select  new  p^ 

4b.  j_f  new  p^  +  old  p^ ,  then  send  DCL(a..  ,IC^ )  to  new  pi  and 

CNCL  to  old  Pj 

4c.  send  MSG(a.. )  to  old  pi  ;  Vt'eGi ,  set  N^(t')  «-  0 

5.  For  CNCL(i),  set  z^t)  ♦  0 

6.  For  DCL(£,a,IC)  set  z^t)  1  , 

6a.  1_f  IC  <  ICj,  then  send  to  I  contents  of  LIST^  from  IC+1  to  IC^ 

while  Incrementing  ICj(t)  up  to  IC1 
6b.  else  IC1(t)  -  IC 

7.  For  B(i) 

7a.  1_f  t=p. ,  then  IC^  «■  IC^  +  1 ,  include  B  in  LIST^, 

7b.  VjeG^  s.t.  zt(J)»l,  ICi ( j)  <  ICr  then 

7c.  send  B  to  j,  IC^(j)  «-  ICj(j)  +  1 

Our  basic  assumption  is  that  all  messages  sent  on  a  link  arrive  in 
arbitrary  but  finite  time  after  their  transmission,  with  no  errors  and  in  the 
correct  order  (FIFO).  Observe  that  this  does  not  preclude  channel  errors 
provided  there  is  an  acknowledgement  and  retransmission  protocol  on  the  link. 
The  proof  that  under  this  condition  the  CRB  protocol  is  Indeed  complete 
appears  in  Appendix  A.  Here  we  only  mention  that  the  important  property 
leading  to  completeness  is  the  statement  of  Lemma  A1 ,  that  will  be  called  the 
session  property.  Broadcast  protocols  associated  with  other  routing  algorithms 
can  be  made  to  have  this  property,  but  several  additions  to  the  algorithm  are 
necessary.  It  is  a  special  feature  of  the  routing  protocol  of  [2]  that  the 
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session  condition  holds  with  no  extrainstructions.  Also  observe  that  as  will 
be  seen  in  Lemma  A2  and  Theorem  A1 ,  completeness  is  achieved  without  requiring 
messages  to  carry  their  counter  number. 

2.3  Finiteness 

Completeness  means  that  broadcast  messages  are  accepted  at  nodes  in 
the  correct  order  and  with  no  duplicates  or  messages  missing.  However,  it  does 
not  ensure  that  all  messages  are  indeed  accepted  at  all  nodes.  The  following 
scenario  shows  that,  since  we  allow  arbitrary  propagation  time  for  messages 
on  each  link,  there  may  be  a  situation  In  the  CRB  algorithm  where  a  node  i 
accepts  no  messages  from  a  certain  time  on.  Considering  Fig.  1,  recall  that 
tQ  is  the  time  when  <4>  of  cycle  a  is  executed  at  node  i,  but  observe 
that  MSG (a)  was  sent  from  i  to  j  when  1  executed  <3>.  Therefore  MSG(a)  may  have 
arrived  to  j  well  before  tq  and  if  the  propagation  time  of  DCL(a.IC)  is  long 
enough,  j  may  have  completed  cycle  a  and  entered  cycle  (a+1)  before  time  tq. 
When  It  entered  cycle  (a+1),  node  J  has  sent  MSG(a+l)  to  1 ,  so  that  node  1 
could  have  performed  <3>  and  <4>  of  cycle  (a+1)  before  time  t.  Now  if  this 
happens  and  at  the  time  of  performing  <4>  of  cycle  (a+1)  node  i  changes  its  p.. , 
then  pi  [t]  f  j  and  B  is  not  accepted.  In  principle,  this  scenario  can  be 
repeated  indefinitely,  so  that  B  and  the  broadcast  messages  following  it 
keep  arriving  at  node  i,  but  will  never  be  accepted. 

In  order  to  correct  the  situation  and  achieve  finiteness,  we  introduce 
an  "Impeding  Mechanism"  in  the  CRB  algorithm.  Control  messages  MSG(a)  sent  from 
j  to  1  will  carry  in  addition  a  variable  z  *  z.(i),  and  MSG(a,z)  such  that 

J 

a=a.j+l,  z=0  received  from  j3p^  will  be  Ignored.  Moreover,  if  node  j  receives 
DCL(a,IC)  with  a<ai  (and  then  by  Lemma  A3  we  have  0=0^1)  node  j  transmits 
again  MSG(aj,z)  with  z*l.  In  this  way,  node  i  postpones  execution  of  <3> 
until  It  receives  acknowledgement  from  j»pj  (in  the  form  of  MSG(aj,  z=l)),  that 
the  last  OCL  message  has  been  received  at  j. 
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For  each  broadcast  message  accepted  at  a  node  i,  it  is  convenient  at 
this  point  to  indicate  explicitly  the  cycle  during  which  it  was  accepted.  To 
do  so  we  replace  LIST^  by  a  set  of  buffers  LIST. (a),  a=1,2,...  (for  the 
meantime  an  infinite  number  of  unbounded  buffers)  and  all  broadcast  messages 
accepted  while  i  was  in  cycle  a  are  stored  in  LIST^a).  Also,  counters 

(a)  are  used,  counting  messabes  accepted  during  cycle  a.  Out  of  the  messages 
corresponding  to  cycle  a,  those  that  have  been  accepted  at  neighbor  «.  as  far 
as  i  knows  are  counted  in  C.(z)(a).  Consequently,  the  counter  IC  is  redefined 
as  the  pair  IC  =  (a,C(a)),  where  IC  <  IC"  means  that  either  a‘  <  a"  or 
a 1  =  a"  and  C'(a')  <  C"(a‘). 

The  resulting  algorithm  is  given  below  and  the  proof  that  it  is  complete 
and  finite  appears  in  the  Appendix. 

The  Reliable  Routing-Broadcast  Algorithm  (RRB)  for  node  i 

1.  For  MSG(t,a,2) 

2.  Vf  l/p^  then  N^U)  1 

3.  Vf  i=p.  and  z=l,  then  :  N.U)-*-l,  <*^a^+l,  send  MSG^  ,zi  [i ) )  to  all 

CeG^  except  p. 

4.  vf  V* ,  holds  Ni (z 1 )  =  1 ,  then: 

4a.  select  new  p^ 

4b.  vf  new  pi  t  old  p^ ,  then  send  DCL^  £ .  (c^  )  to  new  pi  and 

CNCL  to  old  pi 

4c.  send  MSG^)  to  old  p.  ;  Vz'eG^  set  N.(z')  «-  0 

5.  For  CNCL(t),  set  z^U)  0 

6.  For  DCL(l.a.C),  set  z^(l)  *■  1 

6a.  Vf  C<C ■  (a) ,  then  send  to  i  contents  of  LIST^a)  from  C  to  C(a)  while 
incrementing  C-(z)(a)  to  C^(a) 

6b.  Vf  a=  a^-1 ,  then  send  MSG(a.j ,  z to  i 

send  to  z  contents  of  LIST^(a^)  from  1  to  C^a^)  while 
incrementing  C.(i)(a.)  to  C^a^) 
else,  if  Ci  C^(a),  then  C.(t)(a)  <-  C 

_ _ _ _ _ _ _ „  ;  _ _ _  ^ _ -  .•  _ _ 


6c. 
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7.  For  B(i) 

7a.  1_f  4=?^ ,  then  (a^ )  ^C^(a^)  +1 ,  Include  B  in  LIST^a^), 

7b.  VjeGi  >  s.t.  21  ( j)=l ,  Ci  ( at1 ) ( j )  <  C-^. ),  then 

7c.  send  B  to  j,  C.Ca^U)  -  C.(a.)(j)  +  1 

Before  proceeding,  we  note  here  that  the  Impeding  Mechanism  slows 
down  the  routing  algorithm,  but  only  in  extreme  situations.  This  is 
because  the  Impeding  Mechanism  Is  in  fact  activated  only  in  the  case  when 
DCL(a.C)  sent  by  a  node  i  to  j  arrives  there  after  node  j  has  performed  <3> 
of  cycle  (a+1).  Since  such  a  DCL  message  is  sent  by  i  when  it  performs  <4> 
of  cycle  a  ,  this  means  that  propagation  of  DCL  on  link  (i,j)  takes  more 
time  than  propagation  of  the  routing  cycle  a  from  i  all  the  way  to  SINK 
plus  propagation  of  cycle  (a+1)  all  the  way  from  SINK  to  node  j.  This  may 
indeed  haopen  If  we  allow  arbitrary  delays  on  links,  but  the  chances  are 
small . 
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3.  THE  RELIABLE  BROADCAST  PROTOCOL 

The  final  form  of  the  broadcast  protocol  will  be  obtained  from  the 
RRB  algorithm  after  making  several  observations. 

a)  The  broadcast  messages  accepted  by  node  i  while  it  is  in  cycle  a 
are  exactly  those  broadcast  messages  released  by  SINK  while  it  is  in 
cycle  a  (follows  from  Corollary  Al). 

b)  If  node  i  is  in  cycle  a  ,  it  will  never  be  required  to  send  to 
neighbors  messages  accepted  prior  to  cycle  (a-1)  and  therefore  it  needs 

to  store  only  messages  accepted  during  the  present  and  the  previous  cycles. 
From  a)  and  b)  follows  that  we  can  make  significant  simplifications 
in  RRB.  The  variables  a, a.  can  be  binary;  only  two  lists  LIST^O)  and  LIST^l) 
need  to  be  stored;  if  SINK  is  allowed  to  send  no  more  than  M  broadcast 
messages  per  cycle,  those  LIST'S  can  have  finite  size  M;  only  counters  C^O), 

Ci (0(0),  C.(l),  C^(t)(l)  are  needed  and  all  those  are  bounded  by  M;  control 
messages  MSG  need  not  carry  the  variable  a.  The  resulting  broadcast  algorithm 
has  the  followinq  properties: 

Properties  of  RRB  (network  has  N  nodes  and  E  link) 

1)  Reliability 

2)  Finite  memory  and  counters 

3)  No  overhead  cost 

4)  Control  communication  cost:  the  routing  protocol  reouires  2E  messages 
MSG  per  cycle  whether  broadcast  is  operating  in  the  network  or  not. 
Broadcast  requires  no  new  MSG  messaaes,  except  in  the  peculiar  situation 
described  at  the  end  of  Section  2.3.  In  addition  we  need  at  most  N  DCL 
messages  and  N  CNCL  messages  per  cycle. 

5)  Broadcast  communication  cost:  most  of  the  time  broadcast  messages 
propagate  on  spanning  trees.  The  only  situation  when  two  copies  of  the 
same  message  arrive  at  a  node  (and  one  is  ignored)  is  when  a  broadcast 
message  "crosses  paths"  with  a  CNCL  message.  This  means  that  CNCL  is 
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sent  by  i  to  j  and  the  broadcast  message  is  sent  by  j  before  CNCL  has 
arrived  and  is  received  by  i  after  CNCL  was  sent.  The  worst  case 
gives  2(N-1)  messages  in  the  net  per  broadcast  message,  but  in  most 
cases  this  situation  will  not  occur,  especially  if  the  propagation 
time  of  CNCL  is  small,  so  that  the  average  is  very  close  to  (N-l) 
copies  per  message,  which  is  the  minimal  broadcast  cormunication  cost. 

6)  Delay:  the  routing  algorithm  tends  to  find  paths  with  small  total 

weight  (sum  of  link  weights  from  nodes  to  SINK).  The  delay  of  broadcast 
messages  will  be  small  if  the  weights  are  link  delays  and  the  traffic  is 
symmetric  on  links  or  if  the  weights  of  link  (i,j)  contain  a  measure  of 
the  delay  on  link  (j.i). 
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Appendix  A 

Here  we  prove  that  the  CRB  Protocol  of  Section  2.2  is  indeed 
complete  and  that  the  RRB  Protocol  of  Section  2.3  is  indeed  complete  and 
finite.  First  we  recall  several  properties  of  the  routing  protocol  of 
[2]  indicated  in  Section  2.1  and  introduce  several  definitions: 

a)  in  each  cycle  a,  the  routing  protocol  requires  each  node  i  to  send 
exactly  one  MSG(a)  to  each  neighbor 

b)  cycle  a  starts  when  SINK  sends  HSG(a)  to  all  neighbors  (<3>  in  the 
algorithm  for  SINK)  and  ends  when  SINK  receives  MSG(a)  from  all 
neighbors  (line  <4>). 

c)  a  node  i  is  said  to  be  in  cycle  o  while  a^a,  i.e.  from  the  time  it 
perf;  -ms  <3>  with  and  until  it  performs  <3>  with  a.j-a+1 . 

d)  just  before  node  i  performs  <3>,  holds  a=a..+l,  so  that  always 
increases  by  1. 

e)  whenever  we  need  to  indicate  the  value  of  a  variable,  say  p^ ,  at  a 
certain  time  t  we  shall  write  p^[t]. 

Lemma  A1  (Session  Property) 

Consider  the  CRB  Protocol  of  Section  2.2.  If  a  broadcast  message  B 
Is  received  at  time  t  at  node  i  from  j  and  it  is  accepted,  then  B  was 
sent  by  j  after  receiving  the  last  DCL  message  sent  by  i  until  time  t. 

Proof 

Let  t <t  be  the  time  B  was  sent  by  j.  Since  broadcast  messages  are 
accepted  only  from  fathers  (see  <9>  of  CRB)  and  sent  only  to  sons  (see  <7> 
and  <10>),  we  have  p^  [t]  »  j  and  Zj(1)[tl  »  1.  Thus  the  last  DCL  message 
sent  by  i  before  time  t  (at  time  tg  say)  was  Indeed  sent  to  j  and  we  want  to 
show  that  it  was  received  by  j  (at  time  tg  say)  before  time  t,  or  in  other 
words  i  is  the  son  of  j  at  time  t  as  a  result  of  this  last  DCL  and  not  of 
some  previous  DCL's.  This  Is  exactly  the  session  property.  The  timing 
diagram  Is  given  In  Fig.  1.  Consider  also  the  last  CNCL  sent  by  1  before  t 


to  j  and  let  tg,  r^.a  be  respectively  the  time  it  was  sent,  the  time  it  was 
received  and  the  cycle  number  of  i  at  time  t^.  Clearly  tc  <  tQ  and  by  FIFO 
we  also  have  <  tg.  In  order  to  prove  the  lemma  we  need  to  show  that 
Tq  <  t.  Observe  now  that  z j ( i )  =  0  between  x^  and  xg  and  since  Zj(i)[x]  =  1 
time  r  cannot  be  between  xg  and  Xg.  It  Is  sufficient  therefore  to  show  that 
t ^  <  t.  Observe  that  <4b>  shows  that  CNCL  is  sent  after  receiving  MSG(a) 
from  all  neighbors,  in  particular  j  and  before  sending  MSG(a)  to  j  and  there¬ 
fore  a •  fxr]  =  a,  where  a.  is  the  cycle  number  of  node  j.  Suppose  now  that 
Tj.  >  t.  Then  oj[x]  <  a  and  B  was  sent  (and  received,  by  FIFO)  from  j  to  i 
before  MSG(a+l),  so  that  i  could  not  have  performed  <4>  of  cycle  a+1  before  t 
Since  changes  only  in  <4>,  It  follows  that  p. [t]  *  Pi [tc+]  t  j  which  Is  a 
contradiction.  This  proves  the  session  property  of  the  Routing-Broadcast 
Protocol  of  Section  2.2.  Observe  that  the  proof  relies  heavily  on  the  Proper¬ 
ties  of  the  Routing  Protocol  of  [2]. 

Lemma  A2 

If  broadcast  message  B  is  received  at  node  i  from  j  and  is  accepted, 

DO  D 

then  IC“  =  ICj.  (Recall  that  IC?  denotes  the  value  of  the  counter  IC1-  just 
after  node  i  has  accepted  B). 

Proof 

Consider  the  notations  of  Lemma  A1  and  of  Fig.  1.  From  line  <4b> 
in  the  CRB  algorithm  follows  that  the  DCL(a,IC)  message  carries  the  counter 
number  IC  =  IC^ftg].  Since  p^  *  j  on  the  interval  (tQ,  t) ,  node  i  accepts 
during  this  time  broadcast  messages  only  from  j,  and  by  the  Session  Property, 
those  are  sent  only  after  time  xg  at  which  j  performs  <6>,  <7>.  Now  it  is 
easy  to  check  (see  <7>,  <9>-<ll>  for  node  j)  that  In  both  cases, 

IC  <  ICjItq-]  and  IC  >.  lCj[Tg-],  node  j  will  consecutively  send  to  1 
after  xg  the  broadcast  messages  corresponding  to  counter  numbers  IC+1 ,  IC+2, 
etc.  When  they  will  be  received  and  accepted  at  1,  the  counter  IC^  will  be 
increased  respectively  to  IC+1,  IC+2,  etc. 


I 
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Theorem  A1 

The  CRB  algorithm  of  Section  2.2  Is  indeed  complete,  namely 

D  D 

ICJ  3  1 cs ink  ho1ds  for  every  node  *  and  every  broadcast  message  B. 

Proof 

If  the  above  relation  does  not  hold,  let  i  and  B  be  the  node  and 
broadcast  messabe  for  which  it  is  violated  for  the  first  time  throughout 
the  network,  and  let  t  be  the  time  B  was  accepted  at  i.  If  B  was  received 

D  D  DO 

from  j,  then  lemma  A2  implies  ICj  *  IC^  so  that  I Cj  f  IC^i^.  But  B  was 
accepted  at  j  before  being  accepted  at  1,  violating  the  fact  that  the 
statement  of  the  Theorem  held  throughout  the  network  until  time  t. 

For  future  reference  we  need 
Lemma  A3 

If  DCL(o.IC)  arrives  at  node  j,  then  o*aj  or  a j  -  1 . 

Proof 

Consider  the  notations  of  Lemma  A1  and  of  Fig.  1.  Then  c^ftg]  =  a 
and  therefor®  MSG(a+l)  will  be  sent  from  i  to  j  after  the  DCL  message. 
Consequently  <4>  of  cycle  (a+1)  can  be  performed  at  j  only  after  Tg,  hence 
a j [ 1  J  a+1.  On  the  other  hand  tg  is  the  time  i  performs  <4>  of  cycle  a 
and  hence  MSG(o)  has  been  received  at  1  from  j  before  or  at  tg,  so  that 
aj[Tg]  *  a. 

We  next  proceed  to  the  proof  that  the  RRB  Protocol  of  Section  2.3  is 
complete  and  finite. 

Lemma  A4 

In  the  RRB  Protocol,  If  a  MSG(o‘,  z®0'/  arrives  at  i  from  j«p^, 

(and  by  <2>,  <3>  Is  Ignored),  then  MSG(a',  z*l)  will  arrive  at  i  in  finite 
time  from  j  and  then  j  will  still  be  the  father  p^  of  1. 

Proof 

With  the  notations  of  Fig,  1,  where  B  is  replaced  by  MSG(a',  z®0), 
holds  t  <  Tg  (since  z»0)  and  t  >  tg  (since  p^j).  Now 
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<jj ( tq J  i  «j[T]  *  a'  *  [t]+Ho^  [tjj]+1  ■  a+1 ,  where  the  second  equality  follows 

from  property  d)  at  the  beginning  of  the  Appendix.  From  Lemma  A3  follows  that 
[tg]  =  a+1  and  hence  j  will  send  to  1  at  time  td  control  message 
MSG(a‘ ,  z=l)  according  to  line  <6b>  in  RRB. 

Definition 

A  control  message  MSG(a,  z=l)  is  said  to  be  "accepted"  at  node  i 
if  it  triggers  performance  of  <3>  in  RRB  at  node  i.  Also,  define  the  counter 
number  associated  with  an  accepted  message  MSG(a,  z=l )  as  IC^MSGU,  z=l ) )  = 

(a,  Cj (a)=0) . 

Lemma  A5 

With  the  above  definitions,  control  messages  with  z=l  propagate  in 
RRB  as  if  they  were  regular  broadcast  messages. 

Proof 

Broadcast  messages  are  accepted  at  i  only  if  they  arrive  from  p.  and 
are  sent  to  sons,  either  when  they  are  accepted  or  in  response  to  DCL  with 
IC  <  ICj.  Control  messages  MSG(a,  z»l)  are  accepted  only  If  they  arrive  from 
and  are  sent  to  sons,  either  when  they  are  accepted  (<3>  in  RRB)  or  in 
response  to  DCL  with  IC  <  ICi  (<6b>  in  RRB).  Moreover,  MSG(a,  z=l)  is 

D 

accepted  at  i  before  all  broadcast  messages  B  with  IC"  *  (o.  C.(o))»  since 

node  i  enters  cycle  a  as  a  result  of  accepting  MSG(a,  z*1 )  from  pi  and  broad- 

□ 

cast  messages  with  IC^  as  above  are  all  accepted  while  i  is  in  cycle  a.  Now, 
MSG(a,  z*l)  Is  sent  to  any  node  before  all  such  broadcast  messages  (see  <3> 
and  <6b>),  so  that  the  order  Is  preserved  as  well.  Hence  the  statement  of  the 
Lemma. 

Corollary  A1 

The  combination  of  broadcast  messages  and  control  messages  with  z«l 
performs  a  jointly  complete  algorithm,  i.e.  all  such  messages  are  accepted 
In  the  order  released  by  the  source  node  SINK,  with  no  duplicates  and  no 
messages  missing. 


V 
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Theorem  A2 

The  RRB  protocol  Is  complete  and  finite. 

Proof 

From  Lemma  A4  and  the  fact  that  every  routing  cycle  of  the 
algorithm  of  [2]  propagates  in  finite  time,  follows  that  the  propagation 
of  control  messages  with  z=l  is  finite,  namely  every  node  enters  every 
cycle  in  finite  time.  By  Corollary  A1 ,  all  broadcast  messages  released  by 
SINK  while  SINK  is  in  cycle  a  are  accepted  at  each  node  while  the  node  is 
in  cycle  a,  and  since  each  node  enters  cycle  (a+1)  in  finite  time,  all 
such  broadcast  messages  are  accepted  at  each  node  in  finite  time. 
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